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ABSTRACT 



The purpose of this study was to determine whether children 
living in North Carolina who attended child care centers that participated in 
many Smart Start quality improvement efforts have better skills when they 
enter kindergarten than do a comparison group of children from other child 
care centers of family child care homes. Subjects were 214 Smart Start 
children and 294 comparison children. Within the group of Smart Start 
children, a subgroup of 142 were identified who attended centers 
participating in activities directly related to improving child care quality. 
Information on the cognitive, language, and social skills of all subjects was 
gathered as they began kindergarten. Findings indicated that when all 214 
Smart Start children were compared with all comparison children, the skills 
of the two groups were not different. However, the Smart Start-direct 
subgroup did have significantly better cognitive and language skills than 
comparison children. Also, fewer children in this subgroup were rated by 
their kindergarten teachers has having behavior problems than children in the 
comparison group. The findings of this multi-county study support earlier 
single-county reports of the positive effects of Smart Start on children's 
outcomes. The findings suggest that Smart Start efforts need to be directly 
related to improving the quality of child care if they are to have an effect 
on children's school entry skills. (KB) 
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A Six-County Study of the Effects of Smart Start 
Child Care on Kindergarten Entry Skills 

EXECUTIVE SUMMARY 



The purpose of this study was to determine whether children who attend child 
care centers that participated in many Smart Start quality improvement efforts have 
better skills when they enter kindergarten than do a Comparison group of children from 
other child care centers or family child care homes. We recruited 214 Smart Start 
children and 294 Comparison children for this study. Within the group of 214 Smart 
Start children, we identified a subgroup of 142 children who attended centers 
participating in activities directly related to improving child care quality (referred to as 
the Smart Start-Direct subgroup). In the fall of 1998, we gathered information about the 
cognitive, language, and social skills of these children as they began kindergarten. 

When ail 214 Smart Start children were compared with all Comparison children, 
the skills of the two groups were not different. However, the Smart Start-Direct subgroup 
of children did have significantly better cognitive and language skills than Comparison 
children when they entered kindergarten. Also, fewer children in the Smart Start-Direct 
subgroup were rated by their kindergarten teachers as having behavior problems than 
children in the Comparison group. 

The findings of this multi-county study support earlier single-county reports of the 
positive effects of Smart Start on children’s outcomes. The findings suggest that Smart 
Start efforts need to be directly related to improving the quality of child care if they are to 
have an effect on children’s school entry skills. In the interest of being comprehensive, 
local Smart Start partnerships may distribute multiple, diverse services to the child care 
community. This approach may not produce the intended improvements in child care 
quality or child outcomes. To affect school entry skills, the type — not just quantity — of 
Smart Start support is important. 
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Introduction 



North Carolina’s Early Childhood Initiative (Smart Start) was created in 1993 as a 
partnership between state government and local leaders, service providers, and families 
to better serve children under six and their families. The state distributes funds to county 
partnerships, non-profit corporations established specifically for the purpose of 
administering Smart Start activities. The primary goal of Smart Start is to ensure that all 
children enter school healthy and prepared to succeed. One of the ways in which local 
partnerships are working to achieve this goal is by improving the quality of center-based 
child care. Approximately one-fourth of Smart Start funds (averaged across all counties) 
are being spent on child care quality improvement activities such as on-site technical 
assistance (e.g., a consultant visits the center and provides center-specific or 
classroom-specific suggestions for improving the quality of care), programs to increase 
the education and knowledge of early childhood teachers, special enrichment activities 
for children, workshops and CPR training for teachers, and grants to improve facilities 
and curricula. 

The rationale for the child care improvement activities has come from both recent 
research and North Carolina’s relatively poor standing on indices of child care quality. 
Research in early childhood education has demonstrated the importance of high quality 
early childhood education and care in preparing preschoolers for school success, the 
primary goal of Smart Start (see Bryant, Burchinal, Lau, & Sparling, 1994; Burchinal, 
Roberts, Riggins, Zeisel, Neebe, & Bryant, in press; Cost, Quality, & Outcomes Study, 
1995; Frede, 1998; Howes, Phillips, & Whitebook, 1992; Lamb, 1997; NICHD Early 
Child Care Research Network, 1997; Peisner-Feinberg & Burchinal, 1997). 
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Unfortunately, many children in North Carolina do not receive high quality early 
childhood education and care. In a four-state study of child care quality conducted 
before Smart Start began, North Carolina had the highest percentage of centers 
providing poor quality care (Cost, Quality, & Outcomes Study, 1995). Approximately 
189,000 children are enrolled in licensed center-based care in NC (Division of Child 
Development, 1999), about 30% of all NC children under age 6 (Annie E. Casey 
Foundation, 1998). The goal of many local Smart Start efforts is to improve the quality 
of child care in their community. 

In earlier studies conducted as part of the Smart Start evaluation. North Carolina 
children who attended organized child care were rated by their kindergarten teachers as 
having better skills than children who did not attend organized child care (FPG-UNC 
Smart Start Evaluation Team, 1997, 1998). Smart Start has also been shown to be 
related to the improvement in the quality of center-based child care between 1994 and 
1996 (Bryant, Maxwell, & Burchinal, in press). These two studies suggest that 
preschoolers who attend a child care center participating in several Smart Start-related 
child care quality improvement efforts should be better prepared for kindergarten. 
(Throughout the rest of this report these centers will be referred to as Smart Start 
centers. This shorthand term does not imply that these centers are fully funded by 
Smart Start, only that they are participating in Smart Start-funded activities.) 

Child care quality improvement efforts are clearly intended to improve child care 
so that, in the long-term, children who attend those centers are more adequately 
prepared for school. Two previous single-county studies have shown such an effect of 
Smart Start on kindergarten entry skills. In a study conducted in Mecklenburg County, 
children who attended child care centers that received Smart Start quality improvement 
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supports had better skills when they entered kindergarten than did children who had not 
attended such centers. The difference in favor of children who had attended Smart Start 
centers was small but statistically significant and was found only if the children had 
attended the center for three years (Praxis, 1998). 

In a study conducted in Orange County, the effect of attending a Smart Start 
participating center was positive and significant but was evident only for children from 
low-income families, not children from middle-income families. Within the group of 
Orange County children from low-income families, those who attended Smart Start 
centers had much better skills at school entry than children who attended child care 
programs in the general Orange County sample. This study was limited by a small 
sample size (FPG-UNC Smart Start Evaluation Team, 1998). 

The purpose of the study presented here was to determine whether similar 
results would be found in a larger, more geographically diverse sample of children. That 
is, using a large sample of NC children, we wanted to know whether children who 
attended Smart Start child care centers have better skills when they enter kindergarten 
than do children who had other child care arrangements. 

Study Description 

Overview 

In the early summer of 1998, we identified seven diverse partnerships that were 
supporting several child care quality improvement efforts. Of these seven, the Executive 
Directors of six agreed to participate in this study. Together with local partnership staff, 
we identified child care centers that had participated in most of the Smart Start quality 
improvement supports offered by the partnership. Local partnership staff then worked 
over the summer with staff at those child care centers to recruit all children who would 



O FPG-UNC Smart Start Evaluation Team 



3 



be attending kindergarten in the fall. At the beginning of the 1998-99 school year, we 
worked with schools to locate these children in school, asked their kindergarten 
teachers to participate in the study, and recruited a Comparison group of children within 
the same kindergarten classrooms of the Smart Start children. 

Once we identified both Smart Start and Comparison children, we gathered 
information from teacher reports about each child’s cognitive, language, social, and 
behavioral skills. We did not tell the teachers which children were Smart Start and which 
were Comparison children. Although teachers may have known which children attended 
center-based child care, they would not have known whether Smart Start supported 
each center, nor the amount or type of support. Therefore, it is very unlikely that 
teachers rated children differently because they knew which children were in each 
group. 

To gather information about children’s receptive language skills, FPG/UNC 
research assistants conducted one-on-one child assessments in the schools. The 
research assistants did not know whether children were in the Smart Start or 
Comparison group. More detailed information about participating partnerships, child 
care centers, and children is provided below. 

Participating Partnerships 

Two Year 1 partnerships (Burke and Cumberland) and four Year 2 partnerships 
(Chatham, Durham, Forsyth, and Person) from the central and western parts of the 
state participated in this study. Each participating partnership offered a range of quality 
improvement supports to child care centers, such as on-site technical assistance, health 
and safety training and grants, and educational scholarships for teachers. As with any 
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locally determined initiative like Smart Start, the quantity and types of quality 
improvement supports likely varied across partnerships. 

Participating Child Care Centers 

All child care centers in this study participated in many different Smart Start 
technical assistance activities. The list of different activities seemed to be of two types- 
either directly related to improving the quality of care or supportive activities that, 
although useful, were not directly related to improving day-to-day quality of care 
(hereinafter referred to as Direct or Supportive). We were particularly interested in the 
activities that were more directly related to quality of care because of the previous 
research that has shown that if the quality of care is higher, then child developmental 
outcomes should be better (see research summarized earlier). 

Types of activities that have been shown to improve quality, and therefore met 
our definition of Direct, include on-site technical assistance through observations and 
feedback to teachers (Sparks, 1986; Wade, 1984-85) and higher levels of teacher 
education (Phillipsen, Burchinal, Howes, & Cryer, 1997; Whitebook, Howes, & Phillips, 
1989). The first two authors reviewed each Smart Start child care activity and by 
consensus assigned each to either the Direct or Supportive category. Table 1 includes 
list of the different types of activities that were labeled Direct or Supportive and 
Appendix A includes a description of each type of child care activity. Centers that 
participated in ail of the Smart Start-Direct activities offered by their local partnership 
were included in the Smart Start-Direct subgroup. 
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Table 1. List of Smart Start-Direct and Smart Start-Supportive Child Care 
Activities 



Direct 


Supportive 


Enhanced subsidies for higher child care quality 


CPR training 


Enhanced subsidies for higher teacher education 


Developmental screenings 


License upgrades 


Director administrative training 


On-site technical assistance 


Enrichment activities 


Quality improvement and facility grants 


Expansion and start up grants 


TEACH® 


Health and safety assessments 


Teacher education scholarships 


Playground safety 


Teacher salary supplements 


Teacher substitutes 

Transportation 

Specialists 

Subsidies (not tied to quality) 
Workshops 



Participating Children 

Child care directors from the participating Smart Start centers sent recruitment 
letters home to the parents of all children who were expected to attend kindergarten in 
the fall of 1998. We included in our study those children whose parents consented and 
who had attended the Smart Start center for at least 8 months. A total of 214 Smart 
Start children were recruited. On average, these children had spent the last 25 months 
at their center (length of time ranged from 8 to 60 months). When these children entered 
kindergarten, we asked their teachers to send home parent consent letters for all of the 
children in the class. Teachers did not identify children as being in either the Smart Start 
or Comparison group. (See Appendix B for both types of parent consent letters.) 

Through this kindergarten classroom recruitment process, we obtained parent 
consent for a large number of children. If the child had attended other child care 
centers (not involved in many Smart Start activities) or a family child care home, he or 
she was included in the Comparison group. If the child received no out-of-home care 
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before school, he or she was not included in the Comparison group as this study was 
not intended to compare the outcomes of children who were in out-of-home care to 
those of children who remained at home. In addition, this school recruitment process 
identified a few children who had attended a Smart Start center. These parents had 
either not received or responded to the earlier recruitment letter sent via the center 
(e.g., family on vacation), but did respond to the recruitment letter via the kindergarten. 
If the child had attended a Smart Start center for 8 months or longer, we included that 
child as part of the Smart Start group. If the child had attended the Smart Start center 
for less than 8 months, we excluded that child from our study because we did not want 
any children in the Comparison group who had attended a Smart Start center (even if it 
was only for a short time). However, it is possible that the Comparison group may have 
included a few children who attended a child care center or family child care home that 
received a small amount of Smart Start support. 

Finally, we limited the total number of children (Smart Start and Comparison) 
from each class to 12 so we would not overburden any teacher with excessive data 
collection demands. This recruitment procedure resulted in a Comparison group of 294 
children, all of whom had attended a center-based child care program not involved or 
minimally involved in Smart Start (79%) or a family child care home (21%) before they 
entered kindergarten. 

We compensated all participants for their help with this study. Teachers received 
$10 for every child they rated. Participating parents and children received a children’s 
book and a chance to win a $100 gift certificate. (We randomly selected one family 
within each partnership to receive a $100 gift certificate.) 
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Descriptive information about participating children is included in Table 2. 
Teachers reported to us each child’s sex and free or reduced price lunch status. We 
used eligibility for free or reduced price lunch as a proxy for poverty. Eligibility for 
reduced price lunch is based on federal poverty guidelines (i.e., to be eligible for free or 
reduced price lunch, family income must be at or below 185% of the poverty level). 

Some analyses compare the entire Smart Start group to the Comparison group 
and other analyses compare the subgroup of children from the Smart Start-Direct 
centers to the Comparison group. Descriptive information about these groups of 
children is included in Table 2. 



Table 2. Description of Participating Children (Number and Percent) 



Characteristic 


Total 

Sample 

(N = 508) 


Smart Start 
(N = 214) 


SS-Direct* 
(N = 142) 


Comparison 
(N = 294) 


Sex 










Girls 


240 (47%) 


102 (48%) 


71 (50%) 


138 (47%) 


Boys 


268 (53%) 


112 (52%) 


71 (50%) 


156 (53%) 


Free/Reduced Price 










Lunch Status 










Yes 


156 (31%) 


62 (29%) 


34 (24%) 


94 (32%) 


No 


352 (69%) 


152 (71%) 


108 (76%) 


200 (68%) 



’‘The children in the SS-Direct group are a subgroup of the children in the Smart Start 
group. 



Measures 

We used several measures of children’s skills. A brief description of each 
measure is included in this section. 

Kindergarten Teacher Checklist (KTC). The KTC is a 36-item rating scale based 
on the Maryland Systematic Teacher Obsen/ation Instrument developed by the 
Maryland Department of Education (Dauber, Alexander, & Entwisle, 1993). On the KTC, 
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teachers rate the child’s cognitive, language, social, and motor skills on a scale of 1 to 5 
with a higher score indicating greater skills. We used the mean total KTC score in our 
analysis. A copy of the measure is included in Appendix C. 

Social Skills Rating Scale (SSRS). The Social Skills Rating Scale (Gresham & 
Elliott, 1990) is a standardized, norm-referenced rating scale that consists of 30 items 
that measure children’s social skills on a scale of 0 to 2, with a higher score indicating 
greater skills; and 18 items that measure problem behaviors on a scale of 0 to 2, with a 
higher score indicating greater problems. In our analysis we used the standard scores 
for each of these areas — social skills and problem behaviors. These standard scores 
have a mean of 100 and a standard deviation of 15. 

Peabody Picture Vocabulary Test (PPVT-III). The Peabody Picture Vocabulary 
Test-Third Edition (Dunn & Dunn, 1997) is an individually administered test of children’s 
receptive language skills. Children are shown a set of four pictures and asked to select 
the one picture that best represents the word spoken by the examiner. Administration 
time averages about 10 to 15 minutes. We used in our analysis the total standard score 
(mean of 100, standard deviation of 15). A higher score indicates greater receptive 
language skills. 

Data Analysis 

Models were fit using hierarchical linear modeling to account for clustering of 
children within kindergarten classrooms. This procedure accounts, statistically, for the 
fact that some teachers rated multiple children in their classrooms. We also wanted to 
account for three variables that could affect children’s outcomes but over which we had 
no control: Smart Start partnership, poverty (free/reduced price lunch or not), and sex 
(boys, girls). Controlling for these factors, we then looked at two sets of group 
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differences: (1) Smart Start vs. Comparison, and (2) Smart Start-Direct vs. Comparison. 
Interaction terms were included initially in all models; these effects were removed from 
the model when they were not statistically significant. 

Findings 



Smart Start Effects 

Using the whole sample and controlling for the effects of poverty and sex, 
children who attended Smart Start child care centers had skills similar to children in the 
Comparison group. In other words, the groups were not statistically different from each 
other. Means and standard deviations on all outcome measures for the Smart Start and 
Comparison groups are presented in Table 3. 

Table 3. Descriptive Statistics for the Smart Start and Comparison Groups on All 
Kindergarten Outcome Measures 





KTC 


PPVT-III 


SSRS-Social 


SSRS-Prob. 

Behavior 


Smart Start 










mean 


4.3 


102.7 


100.8 


98.9 


(standard 

deviation) 


(.54) 


(14.6) 


(15) 


(14.1) 


Comparison 

mean 


4.2 


102.1 


100.2 


100.6 


(standard 

deviation) 


(.56) 


(14.4) 


(15) 


(14.2) 



However, when children in the Smart Start-Direct subgroup were compared with 
the Comparison group, there were group differences in favor of the Smart Start-Direct 
group. (See Table 4.) Controlling for the effects of poverty and sex, children in the 
Smart Start-Direct group had statistically significantly better kindergarten skills as 
measured by the KTC than did children in the Comparison group. There were no 
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statistically significant differences between the groups on the PPVT-III, SSRS-Social 
Skills or SSRS-Problem Behavior. 



Table 4. Descriptive Statistics for the Smart Start-Direct and Comparison Groups 
on All Kindergarten Outcome Measures 





KTC 


PPVT-III 


SSRS-Social 


SSRS-Prob. 










Behavior^ 


SS-Direct 










mean 


4.4 


104.8 


102.7 


96.4 


(standard 

deviation) 


(.47) 


(14.5) 


(15.3) 


(12.8) 


Comparison 










mean 


4.2 


102.1 


100.2 


100.6 


(standard 

deviation) 


(.56) 


(14.4) 


(15) 


(14.2) 




F(1, 326) = 5.36 


F (1,326) = 2.38 


n.s.*^ 


n.s. 




p = .02 


CM 

II 

Q. 






ai 


= .29 


ES = .20 







®Lower scores indicate fewer problem behaviors. 



‘’ES = effect size. The effect size was calculated by subtracting the adjusted mean of 
the Comparison group from the adjusted mean of the SS-Direct group, and dividing by 
the square root of the pooled variance. 

‘^n.s. = non significant 



A difference that is statistically significant at the p = .02 level means that there is 
only a 2% chance that the difference occurred simply due to chance. Although statistical 
significance helps us interpret the probability of finding a difference between groups, it 
does not provide any information about the meaningfulness of the group difference. An 
effect size helps determine the meaningfulness of a difference between groups. The 
effect sizes in this report can be interpreted as the proportion of a standard deviation 
difference between the two groups. An effect size of 1.0 means that the difference 
between the two groups is one standard deviation~a very large, meaningful difference. 
An effect size of .20 means that the difference between the two groups is one-fifth of a 
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standard deviation. Knowing the measures included in our study, we consider an effect 
size of .20 to .30 to be small but meaningful. 

The difference between children in the Smart Start-Direct and Comparison 
groups was statistically significant and meaningful for the KTC. For the PPVT, the 
difference was small and not statistically significant but can be considered meaningful 
based on the .20 effect size. There were no statistically significant or meaningful group 
differences on the mean scores of children's social skills and problem behaviors. These 
findings suggest that children attending centers participating in Smart Start activities 
that are more directly related to quality improvement have better cognitive and language 
skills but not better social skills. 

Another way to examine the data is to compare the proportion of children in each 
group (Smart Start-Direct vs. Comparison) who scored very poorly on the outcome 
measures. Children with scores one standard deviation or more below the mean on the 
KTC (at or below 3.71), PPVT-III (at or below 87.84), or SSRS-Social Skills (at or below 
85.46) were considered as scoring poorly. Because higher scores on the SSRS- 
Problem Behavior indicate more problems, children with scores one standard deviation 
or more above the mean (at or above 1 14.09) were considered as scoring poorly. As 
evident in Table 5, statistically significantly fewer children in the Smart Start-Direct 
group scored poorly on the KTC, PPVT-III, and SSRS-Problem Behavior than did 
children from the Comparison group. These data provide additional evidence for the 
positive effects of Smart Start on children's school entry skills including, in this type of 
analysis, fewer problem behaviors. 



16 






. Six-County Study of Smart Start Effects on Kindergarten Entry Skiiis 



12 



Table 5. Proportion of Children Who Scored Poorly on Each Outcome Measure 





KTC 


PPVT-III 


SSRS-Social 


SSRS-Prob. 

Behavior 


SS-Direct 


9% 

(12 out of 139) 


8% 

(12 out of 142) 


17% 

(23 out of 136) 


10% 

(14 out of 138) 


Comparison 


17% 

(50 out of 289) 


15% 

(45 out of 294) 


16% 

(45 out of 287) 


18% 

(51 out of 287) 




X = 5.7 

p = .02 


X = 4.0 
p = .047 


not 

significant 


X = 4.2 
p = .045 



Effects of Poverty and Sex 

Although the positive effects of Smart Start-Direct child care quality activities 
were seen on children’s outcomes after controlling for poverty and sex, poverty itself 
was a consistent predictor of kindergarten skills. Poverty was measured by free or 
reduced-price lunch status. As shown in Table 6 children from low-income families had 
significantly poorer scores on all outcome measures. As for sex differences, boys and 
girls scored similarly on all outcome measures except the KTC. Teachers rated girls as 
having statistically significantly higher skills than boys — although the difference was 
very small. 
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Table 6. Descriptive Statistics for All Kindergarten Outcome Measures for Poor 
vs. Not Poor Children and Girls vs. Boys 





KTC 


PPVT-III 


SSRS- 

Social 


SSRS-Prob. 

Behavior 


Poverty 

No 


4 4*** 


106.1*** 


103.3*** 


97.5*** 




(.47) 


(13.7) 


(14) 


(13.1) 


Yes 


4.0 


93.6 


94.8 


103.7 




(.63) 


(12.6) 


(15.7) 


(15.2) 


Sex 

Girls 


4.3* 


101.9 


99.7 


99.1 




(.54) 


(14) 


(15.3) 


(13.9) 


Boys 


4.2 


102.4 


101.6 


99.8 




(.57) 


(15.1) 


(14.8) 


(14.3) 



* p < .05 and ES = 

*** p < .001 and ES > 



Conclusions 

The results of this study suggest that Smart Start assistance to child care centers 
helps young children come to school ready to succeed if the assistance is directly 
related to quality improvement. Specifically, children who attended child care centers 
that participated in Smart Start activities directly related to improving quality had better 
cognitive and language skills when they entered kindergarten than did children from 
other child care centers or family child care homes. Additionally, fewer children in the 
Smart Start-Direct group were rated by their kindergarten teachers as having behavior 
problems. 

The findings highlight the importance of the type of Smart Start supports to child 
care centers that are associated with better child outcomes. Children from child care 
centers that received a large number of Smart Start child care supports did not 
necessarily have better skills than children in the non-Smart Start Comparison group. 
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Children’s outcomes were notably better only when the child care center had received 
Smart Start efforts directly related to quality improvement. 



What does this mean for local Smart Start partnerships? The findings of this 
study suggest that child care quality improvements should be intense and based on 
best practice (e.g., demonstrated in the literature to improve quality) if one wants to 
have an effect on children’s school entry skills. Local partnerships should review the 
types of child care supports they are currently funding to ensure that they are promoting 
efforts most likely to bring about the desired changes. In the interest of being 
comprehensive, local Smart Start partnerships may distribute multiple, diverse services 
to the child care community. This approach may not produce the intended 
improvements in child care quality or child outcomes. Funding more direct improvement 
supports for a smaller number of child care centers may be more effective in producing 
changes-although it may be less popular than the alternative, spread-the-wealth 
funding strategy. Finally, partnerships need to measure over time the quality of care 
provided in centers that are participating in Smart Start to determine whether their 
quality improvement efforts are successful. 

The findings and recommendations from this study should not be construed to 
mean that local partnerships should provide none of the activities listed under the Smart 
Start-Supportive category or that supportive activities are not worthwhile. Ensuring that 
all teachers are certified in CPR, for instance, is important for children’s health, but 
should not be expected to raise children’s kindergarten entry skills. Centers that 
participate only in Smart Start-Supportive activities will not likely improve children’s skills 
at school entry. 
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Because many of the centers in the Smart Start-Direct group also participated in 
several Smart Start-Supportive activities, we were not able to compare the effects of 
centers that participated in only Smart Start-Direct vs. only Smart Start-Supportive 
activities. Additional research is needed to determine the impact of particular Smart 
Start activities on the quality of care and children's outcomes. Until that research is 
available, we suggest that local partnerships think carefully about how the kinds of child 
care center supports they provide link to their overall goals for children and families. 

The negative impact of poverty on kindergartners’ skills is another important 
finding from this and previous Smart Start reports. Kindergartners from low-income 
families consistently demonstrate fewer cognitive, language, and social skills than 
children from non-poverty families. National studies like the Cost, Quality, and 
Outcomes study have shown that high quality child care can be especially important for 
improving the school outcomes of at-risk children (Peisner-Feinberg et al., 1999). We 
also saw a stronger effect of Smart Start for poor children in a previous study (FPG- 
UNC Smart Start Evaluation Team, 1998). In this study, though, the effects were similar 
for children from both low-income and middle-income families. It is possible that the 
quality of care provided in Smart Start child care centers still is not as high as needed to 
impact differentially the outcomes of at-risk children. Although local Smart Start 
partnerships may celebrate the fact that Smart Start-Direct quality improvement efforts 
are positively affecting children’s skills in kindergarten, they should continue to improve 
the quality of all child care, especially for children at risk for later school problems. 

As a final note about the impact of poverty, we should realize that although Smart 
Start can positively affect the lives of young children and their families living in poverty. 
Smart Start cannot by itself be expected to overcome all the problems associated with 
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poverty. Local Smart Start partnerships may want to join the efforts of other economic 
and community organizations to address the broader issues related to poverty. 

We must acknowledge the limitations of this study when interpreting the findings. 
The analyses for this study determined associations, not causal links, between Smart 
Start and kindergarten outcomes. As with any study of associations, one must 
recognize the possibility that other, unmeasured factors besides Smart Start produced 
the positive outcomes. We also were not able to manipulate the type of Smart Start 
child care quality improvements that participating centers received (i.e., randomly 
assign centers to receive particular improvement efforts), so we cannot say definitively 
which types of improvement supports were most effective. The findings from our study, 
though, suggest that activities such as on-site technical assistance, enhanced subsidies 
for higher quality, and support for teacher education are effective ways of improving the 
quality of child care and child outcomes, findings that are consistent with previous 
studies in this field. 

We must also recognize that " school readiness" is only one desired outcome of 
Smart Start. Parent involvement and children's health, for example, are also important. 
More broadly. Smart Start aims to impact communities' service systems for young 
children and their families. The FPG-UNC Smart Start evaluation team has conducted 
other studies that address these Smart Start goals. (See Appendix D for a list of 
evaluation reports.) Local partnerships work hard to strike the right balance of programs 
that meet the needs of their communities. 

In conclusion, this study supports earlier findings of the positive effects of Smart 
Start on children’s outcomes. Because the children in this study were from six counties, 
we are much more confident that the results represent Smart Start’s effects across 
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North Carolina. Certain types of Smart Start child care efforts seem to improve the skills 
of young children. The findings of this study point to the importance of funding efforts 
that are directly related to improving the quality of care in child care centers rather than 
sprinkling several activities across a large number of centers. The type, not just 
quantity, of Smart Start support matters. Additional research is needed to better 
understand which particular Smart Start quality improvement efforts produce greater 
changes in classroom quality and child outcomes. 
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Description of Smart Start Child Care Activities 
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Description of Smart Start Child Care Activities 



Direct Quality Improvement Activities 

Enhanced subsidies for higher child care quality. Families with low incomes are 
eligible to receive government assistance for child care expenses. The Division of Child 
Development sets a standard rate for each county (e.g., $400 per month per child). As 
an incentive for providing higher quality care, Smart Start provides child care centers 
with AA licenses or NAEYC (National Association for the Education of Young Children) 
accreditation subsidies that are higher than the standard rate (e.g., an additional $25 
per month per child for a total of $425). 

Enhanced subsidies for higher teacher education. Families with low incomes are 
eligible to receive government assistance for child care expenses. The Division of Child 
Development sets a standard rate for each county (e.g., $400 per month per child). 
Child care centers with teachers meeting a certain educational standard (e.g., 75% or 
more of the teachers have a North Carolina Child Care Credential) receive Smart Start 
subsidy funds that are higher than the standard rate (e.g., an additional $25 per month 
per child for a total of $425). 

License upgrades. Child care centers receive Smart Start funds to make the 
improvements necessary to move from an A- to AA-license or become nationally 
accredited. 

On-site technical assistance. Smart Start funds child care consultants who visit 
child care centers to develop center-specific or classroom-specific suggestions for 
improving the quality of care. 
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Quality improvement and facility grants. Child care centers receive Smart Start 
funds to purchase classroom educational materials, improve the physical space, and 
other things designed to improve the quality of care. 

TEACH® (Teacher Education and Compensation Helps). TEACH® provides 
education scholarships and support for release time for child care teachers. Each 
teacher’s center director agrees to increase the teacher’s salary or provides a bonus 
when the education courses are completed 

Teacher education scholarships. Child care teachers can receive Smart Start 
scholarships to cover the costs of enrolling in a college or community college course in 
early childhood education. 

Salary supplements. Smart Start supplements the salaries of child care teachers 
who have higher education levels. 

Supportive Child Care Activities 

CPR training: Smart Start provides CPR training for child care teachers. 

Developmental screenings. Smart Start provides developmental screenings 
(including hearing, vision, and speech/language screenings) to children at child care 
centers. 

Director administrative training. Smart Start provides training to child care 
directors on administrative issues (e.g., budgeting, personnel issues). 

Enrichment activities. Smart Start supports librarians, art teachers, reading 
teachers, music teachers, and others who travel from center to center to teach special 
enrichment activities to children. 

Expansion and start up grants. Smart Start provides funds to help establish a 
child care center or to expand an already existing one. 
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Health and safety assessment. Smart Start supports assessments of the health 
and safety practices of child care centers, which includes recommendations for 
improvement. 

Playground safety. Smart Start purchases playground equipment for centers and 
helps child care centers fix playground safety hazards. 

Teacher substitutes. Smart Start funds trained teacher substitutes for child care 
centers. 

Transportation. Smart Start funds transportation to bring children from their home 
to their center and back home. 

Specialists. Smart Start funds nurses and therapists (e.g., mental health, speech) 
to consult with teachers and provide services to children in child care centers. 

Subsidies. Families with low incomes are eligible to receive government 
assistance for child care expenses. The Division of Child Development sets a standard 
rate for each county (e.g., $400 per month per child). Smart Start may provide funds so 
that more families receive subsidies (e.g., by reducing the waiting list or raising the 
income eligibility requirement). This general type of Smart Start subsidy is not tied to the 
quality of care or to teachers’ education levels. 

Workshops. Smart Start offers or funds various workshops for child care teachers 
and directors. 
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Appendix B 



Smart Start Group Parent Consent Letter 
Comparison Group Parent Consent Letter 
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Smart Start Group Parent Consent Letter 



Dear Family, 



Will you help us? The child care center that your child attends has done a lot of work with Smart Start to 
improve the quality of the care they provide. We would like to learn more about how Smart Start has 
helped this child core:czn\zr and other centers prepare children for school. If you want to help us with 
this study, please fill out the form and send it back to us in the enclosed stamped envelope within 
the next week. Please keep the yellow copy for your records. If you fill out the form and send it to us, 
you will have a chance to win a $100 Walmart gift certificate. We will also send you a small gift after 
your child has finished the study. (If you have already sent us this form, thank you! You do not need 
to return another form.) 

If your child will be entering kindergarten this fall cmd\f you would like to help us with this study, we 
would like your permission to do three things. 

1. We will ask your child's kindergarten teacher this fall to tell us about your child's learning, social, 
behavioral, and language skills, which are skills that are important for being prepared for school. 

We will also ask her to tell us whether your child is a boy or girl and whether your child 
participates in the school lunch program. 

2. We will come to your child's school this fall and meet one-on-one with your child to play a game-like 
activity to find out about your child's language skills. This activity will take about 15 minutes and 
will be done at a time that is good for your child and your child's teacher. 

3. We will summarize some information from your child's Kindergarten Health Assessment form, 
which is usually kept in the school's records. This information will help us learn more about 
children's health, which is also an important part of being prepared for school. 

You do not have to be in our study. Saying "no" to our study will not affect anything at your child's child 
care center or school, and it won't affect any Smart Start services you or your child are receiving. Saying 
"no" will not affect you or your child in any way. 

Your help is very important to us. We hope you will help us learn more about how Smart Start helps 
prepare children for school. If you have any questions about this Smart Start Kindergarten Study, please 
call Karen Taylor collect at 919-966-2559. 



THANKS FOR YOUR HELP! 



This study is being done by the <insert county name> County Partnership and the FP& Smart Start Evaluation team. 
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Smart Start Group Parent Consent Letter 



I understand the research study as described in the letter and have had all my questions about the Smart 
Start Kindergarten Study answered. I have been given a copy of this letter for my records. I know that 
as a person who is being asked to help with this study, I have rights. If I ever think that these rights have 
been violated — that the researchers have not done the right thing — I can call David Eckerman at 919-962- 
7761 or write to him at Academic Affairs Institutional Review Board, CB #4100, UNC, Chapel Hill, NC 27599- 
4100 or email at aa-irb.unc.edu. 



YES, I would like my child to be in this study. 



Parent's Signature Date Child's First and Last Names (please print) 

If you said YES, please answer these questions. 

1. When was your child born? 

Month Day Year 

2. What is the name of the elementary school where you think your child will go to kindergarten in 
the fall of 1998 and which county is the school in? 



Name of your child's elementary school County 

3. What is the name of the child care center your child now attends (or did attend this past year) 
and which county is the child care in? 



3a. How many years has s/he attended 

Name of your child's child care center this center? 

4. In addition to the chance of winning the $100 Walmart gift certificate, we would like to give 
you a small gift as a way of saying "thanks." Which gift would you like? { Please check one?i 

Please send me and my child: a children's book OR a $5 McDonald's gift certificate. 

5. What is your mailing address (so we can send you the gift)? 



OR 



NO. I do not want my child to be In this study. 



Parent's Signature Date Child's First and Last Names (please print) 



Name of your child's child care center County 
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Comparison Group Parent Consent Letter 



Dear Family, 

Will you help us? We would like to learn more about how local Smart Start efforts have helped 
prepare children for school. We would like you to participate in this study whether you have had 
any involvement with Smart Start or not. 

If you want to help us with this study, please fill out the form and send it back to us in the 
enclosed stamped envelope within the next week. Please keep the yellow copy for your 
records. If you fill out the form and send it to us, you will have a chance to win a $100 Walmart 
gift certificate. If we chose your child to be in our study, we will also send you a small gift after 
your child has finished the study (There is a chance that we will not be able to include your child in 
this study. If that happens, you will still have a chance to win the $100 Walmart gift certificate.) 

If you would like to help us with this study, we would like your permission to do three things. 

1. We would ask your child's kindergarten teacher to tell us about your child's learning, social, 
behavioral, and language skills, which are skills that are important for being prepared for 
school. We will also ask her to tell us whether your child is a boy or girl and whether your child 
participates in the school lunch program. 

2. We will come to your child's school this fall and meet one-on-one with your child to play a game- 
like activity to find out about your child's language skills. This activity will take about 15 
minutes and will be done at a time that is good for your child and your child's teacher. 

3. We will summarize some information from your child's Kindergarten Health Assessment form, 
which is usually kept in the school's records. This information will help us learn more about 
children's health, which is also an important part of being prepared for school. 

You do not have to be in our study. Saying "no" to our study will not affect you, your child, or your 
child's school in any way. 

Your help is very important to us. We hope you will help us learn more about how different types 
of child care help prepare children for school. If you have any questions about this Smart Start 
Kindergarten Study, please call Karen Taylor collect at 919-966-2559. 



THANKS FOR YOUR HELP! 



This study is being done by your local Partnership for Children and the FPG Smart Start evaluation team. 
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Comparison Group Parent Consent Letter 

I understand the research study as described in the letter and have had all my questions about the Smart 
Start Kindergarten Study answered. I have been given a copy of this letter for my records. I know that 
as a person who is being asked to help with this study, I have rights. If I ever think that these rights have 
been violated — that the researchers have not done the right thing — I can call David Eckerman at 919-962- 
7761 or write to him at Academic Affairs Institutional Review Board, CB #4100, UNC, Chapel Hill, NC 27599- 
4100, or at aa-irb©unc.edu. 

□ YES, I would like my child to be in this study. 



Parent's Signature Date Child's First and Lost Names (please print) 

If you said YES, please answer these questions. 

1. When was your child born? 

Month Day Year 

2. Last year (August 1997 - August 1998), was your child cared for by someone other than you for 
more than 10 hours a week? 

□ NO 

□ YES IF YES, WHAT TYPE OF CARE WAS IT> 

□ center. Head Start, preschool CENTER NAME, 

□ care in someone else's home or your home 

□ with 3 or more children NAME OF PROVIDER 

□ with 1 or 2 children 

3. If we choose your child to be in the study, we will give you a small gift as a way of saying 

"thanks." Which gift would you like? { Please check one.') 

Please send me and my child: a children's book OR a $5 McDonald's gift certificate. 

4. What is your phone number and mailing address (so we can send you the gift)? 

Phone number: ( ) 

Address: 

Street City State Zip 

□ NO, I do not want my child to be in this study. 

Parent's Signature Date Child's First and Lost Names (please print 
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Smart Start 



Kindergarten Teacher Checklist 

(based on the Maryland Systematic Teacher Observation Instrument) 



DO NOT BEND 


Sex of child F M 




Child's date of birth 


m m / d d / y y 
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GENERAL PURPOSilE DATA SHEET !i 
form no. 70921 



USK NO :2 PE.NCIL ONlv 



NEVER 
I SELDOM 

SOMETIMES 
OFTEN 
ALWAYS 



r 



1. 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10. 
11 . 
12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20 . 
21. 
22 . 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 
39- 
40. 



A • 



Can copy a circle, square, and triangle so that It is recognizable. 

Gets along with other children in various situations. 

Can tell about a picture while looking at it. 

Names and locates at least five parts of his body. 

Knocks over things when reaching for them. 

Can repeat sentences such as 1 like to play outside” in correct order. 

Cringes or pulls away when approached by others. 

Can recognize own name in print. 

Stays with the activity at hand. 

Can tell about a recent school activity (e.g., field trip). 

Follows directions. 

Fumbles for words, uses a wrong word, or says s/he forgot what s/he was trying to say. 

Drowsy, sleepy, or sleeps. 

Names common objects such as chair, desk, table. 

Fights, shouts, or shakes his/her fist as a preferred means of solving problems. 

Identifies likenesses and differences in pictures, objects and forms. 

Gives own name and age when asked. 

Stares into space. 

Can identify colors (I.e., red, yellow, blue, green) by name. 

Says, “I can’t” when presented with school tasks. 

If child prints, s/he prints words, letters, and/or numbers backwards. 

Hurts children and/or animals for no apparent reason. 

Speech is understandable. 

Works and solves problems independently. 

Destroys or damages things, breaks toys. 

Matches objects to pictures (e.g., toy truck to picture of truck). 

Finishes tasks late. 

Can tell about a story after listening to it. 

Stumbles, trips, or falls. 

Says “huh” or “whaf after s/he has been told something or asked a question. 

Can tell how many objects up to five. 

Classifies objects by categories, such as food or clothing. 

Speaks in sentences of more than three words. 

Discriminates between fine differences in sounds heard (e.g., boy, toy). 

Arranges a three-part picture story In correct sequence. 

Retells story in correct sequential order. q n 

Do you think this child qualifies for free or reduced price lunch? (A=Yes, B=No) ^ ^ 

Does this child have any disabilities? (A=Yes. B=No) 

Has this child been retained in kindergarten? (A=Yes. B=No) 

Did the child attend organized child care before beginning kindergarten? (A=Yes. B=No, C= Don’t Know) 
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List of FPG-UNC Smart Start Evaluation Reports 
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REPORTS FROM THE FPG-UNC SMART START EVALUATION TEAM 



Summary Reports 

Smart Start Evaluation Plan (September 1994) 

This report describes our comprehensive evaluation plan at the onset of the evaluation, designed to 
capture the breadth of programs implemented across the Smart Start partnerships and the extent of 
possible changes that might result from Smart Start efforts. 

North Carolina's Smart Start Initiative: 1994-95 Annual Evaluation Report (June 1995) 

This report summarizes the evaluation findings to date from both quantitative and qualitative data 
sources. 

North Carolina's Smart Start Initiative: 1996-97 Annual Evaluation Report (April 1997) 

This report summarizes evaluation findings related to each of the four major Smart Start goals. 

North Carolina's Smart Start Initiative: 1998 Annual Evaluation Report (January 1999) 

This report summarizes evaluation findings related to each of the four major Smart Start goals. 

Child Care Quality 

Center-based Child Care in the Pioneer Smart Start Partnerships of North Carolina (May 1996) 

This brief report summarizes the key findings from the 1994-95 data on child care quality. 

The Effects of Smart Start on the Quality of Child Care (April 1997) 

This report presents the results of a 2-year study of the quality of child care in the 12 pioneer 
partnerships. 

Child Care in the Pioneer Partnerships 1994 and 1996 (December 1997) 

This report presents more detailed information about child care centers that were included in The Effects 
of Smart Start on the Quality of Child Care (April 1997). 

Effect of a Smart Start Playground Improvement Grant on Child Care Playground Hazards (August 
1998) 

This report presents results from a comparison of the playground safety of child care playgrounds in a 
county that used Smart Start funds for playground improvement compared to a non-Smart Start county. 

KIndergartners’ Skills 

Kindergartners' Skills in Smart Start Counties in 1995: A Baseline From Which to Measure Change 
(July 1997) 

This report presents baseline findings of kindergartners’ skills in the 43 Smart Start counties. 

The Effects of Smart Start Child Care on Kindergarten Entry Skills (June 1998) 

This report presents results from kindergartners who attended Smart-Start-funded child care centers 
compared to a random group of kindergartners who attended a broad range of child care or no child care. 

Collaboration 

Bringing the Community into the Process: Issues and Promising Practices for Involving Parents 
and Business in Local Smart Start Partnerships (April 1997) 

This report describes findings from interviews and case studies about the involvement of parents and 
business leaders in the Smart Start decision-making process. 

Smart Start and Local Inter-Organizational Collaboration (August 1998) 

This report presents data about the effectiveness of the Smart Start initiative on improving collaborative 
relationships. Qualitative and quantitative data were obtained from 269 respondents in 10 local 
Partnerships. 
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Understanding the Smart Start Process 

Emerging Themes and Lessons Learned: The First Year of Smart Start (August 1994) 

This report describes the first-year planning process of the pioneer partnerships and makes some 
recommendations for improving the process. 

Keeping the Vision in Front of You: Results from Smart Start Key Participant Interviews (May 
1995) 

This report documents the process as pioneer partnerships completed their planning year and moved into 
implementation. 

Reinventing Government? Perspectives on the Smart Start Implementation Process (November 
1995) 

This report documents pioneer partnership members' perspectives on 2 major process goals of Smart 
Start: non-bureaucratic decision making and broad-based participation. 

Other 

Effects of Smart Start on Young Children with Disabilities and their Families (December 1996) 

This report summarizes a study of the impact of Smart Start on children with disabilities. 

Families & the North Carolina Smart Start Initiative (December 1997) 

This report presents findings from family interviews of families who participated in Smart Start in the 
pioneer counties. The interviews included questions about child care, health services, family activities 
with children, and community services and involvement. 

Smart Start Client Information System Feasibility Study (September 1998) 

This report presents findings from a study of the feasibility of creating a system to count uniquely all 
children and families served by Smart Start. 

To obtain copies of these reports, please call Marie Butts at (919) 966-4295, or 

Email her at Marie_Butts@unc.edu 

VISIT OUR WEBPAGE AT www.fpq.unc.edu/~smartstart 
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